1. Field of the Invention
The present invention belongs to the field of image processing and computer vision, and particularly relates to a novel image foreground matting method based on neighborhood and non-neighborhood smoothness priors.
2. Background of the Invention
Image foreground matting intends to decompose an image I into a foreground F and a background B. From the mathematical point of view, the image I is a linear combination of F and B in the following manner:C=Fα+B(1−α),where α defines opacity of each pixel, and has a value in a range of [0, 1]. Accurate image matting is of vital importance in different image and video editing applications. However, since the number of unknown points is much larger than that of known equations, the equations cannot be solved. Therefore, generally a method is adopted in which a user brush type interaction is used or a black-white-gray trimap is input to simplify the solution of such a problem.
The existing image methods can usually be divided into three categories: a sampling-based method, an affinity-based method, and a comprehensive method which is the combination of these two methods.
Sampling-based image foreground matting simultaneously estimates α (alpha) value of a pixel as well its foreground color and background color. In various methods, different parametric or non-parametric models are used to sample neighboring pixels of the known foreground area and background area. Ruzon and Tomasi assume that unknown pixels lie in a narrow band area at the edge of the foreground area. Then, this method was extended by Chuang et al. with a Bayesian framework. In case that the unknown pixels are located near the edge of foreground and the number of unknown pixels is relatively small, these methods provide good results. Rhemann et al. proposes an improved color model based on Geodesic distance sampling. In a shared matting method, the pixels are sampled in different directions of light. Generally, these methods have relatively good effects when the color neighborhoods are smooth.
The affinity-based image foreground matting is independent from the foreground color and background color, so that the problem of alpha matting is solved. In the Poisson matting method, it is assumed that the gradient of alpha mattes is proportional to that of image. In the image foreground matting method based on random walk algorithm (random walk matting), the random walk algorithm is used to solved α values according to the neighboring color similarity. In the closed-form matting method, a color line model is assumed on a neighborhood window, and the problem of alpha matting is solved by minimizing a cost function. In the spectral analysis-based image foreground matting method (spectral matting), its relationship with the spectral clustering is explored so that it is extended into an unsupervised method. Laplacian image matting is combined with different data constraints, prior, or learning-based methods to solve the problem of image matting. However, under the assumption of neighborhood smoothness, this method is insufficient to solve a complicated image problem. Therefore, we combine it with the non-neighborhood smoothness prior to improve the results.
The image foreground matting method, which integrates sampling and similarity, makes a good balance between these two methods. In a robust matting method, samples with high confidence degree are firstly sampled, and then the image foreground matting energy is minimized by the random walk algorithm. In a global sampling matting method, the random search algorithm from the PatchMatch algorithm is used to search global optimal samples.
In the closed-form matting, the Laplacian matrix for image foreground matting is obtained from the color line model, and is used for constraining alpha matting within the neighborhood window. This neighborhood smoothness prior can be combined with the data set obtained from color sampling. Such smoothness prior has a good effect in the image area where there are only a constant number of foreground colors and background colors. He et al. uses a generalized Patchmatch method to improve the effect of color sampling. Recent research indicates that the data set and neighborhood smoothness set can be combined to provide high quality results. However, during calculating Laplacian matrix, it is difficult to set a proper neighborhood window. A small window may be insufficient to capture the detail information of structures. On the other hand, a large window may destroy the color line model, which will also lead to bad results.
Recently, Chen et al. has proposed a manifold preserving edit propagation method, and applied it to the transparent image matting. We note that this method in fact relates to a novel alpha matting based on non-neighborhood smoothness prior. In this method, α values of remote pixels are linked together, which is complementary with Laplacian matting. When only this non-neighborhood smoothness prior is applied, the neighborhood structure information of translucent object would not be captured. Thus, we propose to combine this non-neighborhood smoothness prior with neighborhood Laplacian smoothness prior, and include it into an ordinary data set. Our novel image matting algorithm exhibits excellent performance on the standard test data set.